Output Coding and Modularity for Multi-class Problems

We study backpropagation networks learning classiication problems with multiple classes k > 3. The common way to code the output of a network is the one-per-class (OPC) method, where one bit is assigned to each class. A technique called error-correcting output coding (ECOC) converts the k-class learning problem into a large number of two-class learning problems. We propose to use modular architectures as a way to decorrelate the (redundant) network outputs. Various modular architectures are tested on an artiicial problem. We conclude that ECOC only improves upon OPC when combined with a suuciently modular approach.